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Abstract 

The celebrated formula of Schlafli relates the variation of the dihedral angles of a 
smooth family of polyhedra in a space form and the variation of volume. We give a 
smooth analogue of this classical formula - our result relates the variation of the volume 
bounded by a hypersurface moving in a general Einstein manifold and the integral of the 
variation of the mean curvature. The argument is direct, and the classical polyhedral 
result (as well as results for Lorenzian space forms) is an easy corollary. We extend it 
to variations of the metric in a Riemannian Einstein manifold with boundary. 

We apply our results to extend the classical Euclidean inequalities of Aleksandrov to 
other 3-dimensional constant curvature spaces. We also obtain rigidity results for Ricci- 
flat manifolds with umbilic boundaries and existence results for foliations of Einstein 
manifolds by hypersurfaces. 



Resume 

La formule classique de Schlafli relie la variation des angles diedres d'une famille lisse 
de polyedres dans un espace a courbure constante et la variation du volume. 

On donne un analogue regulier de cette formule classique - notre resultat relie la vari- 
ation du volume borne par une hypersurface se deplagant dans une variete d'Einstein a 
I'integrale de la variation de la courbure moyenne. Puis nous I'etendons aux variations 
de la metrique a I'interieur d'une variete d'Einstein riemannicnne. 

Comme application, on donne un resultat de rigidite pour les varietes Ricci-plates a 
bord ombilique, quelques formules concernant les feuilletages de varietes d'Einstein par 
des hypersurfaces, ainsi que des analogues de la formule d'Alexandrov-Fenchel dans les 
varietes de dimension 3 a courbure constante. 



Let M be a Riemannian (m + l)-dimensional space-form with constant curvature and 
{Pt)t£[o,i] a one-parameter family of polyhedra in M bounding compact domains, all having the 
same combinatorics. Call Vj the volume bounded by Pt, Oi^t and Wi^t the dihedral angle and the 
(m — 1)- volume of the codimension 2 face i of Pj. The classical Schlafli formula (see for instance 
|Mil94| or |Vin93|) relates the variation of Vt and of the angles Oi^t as follows: 
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Schlafli's formula: 
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This formula has been extended and used on several occasions recently, see for instance [CR96], 
| SP97 |, I Bon ], The first goal of this paper is to extend (|l|) to deformations of smooth hypersurfaces 
in if". A remarkable point is that the formula which appears remains valid for deformations 
of hypersurfaces in Einstein manifolds. In section 2, we extend it also to a much more general 
situation, namely the deformations of the Einstein metric in an Einstein manifold with boundary. 

When the Wi,t are constant, the left-hand side of (|^) is a polyhedral analogue of the variation 
of the mean curvature integral of a hypersurface. Indeed, the polyhedral analogue of the mean 
curvature integral is _ff = ^ WiOi (where 9i is the ea;terior dihedral angle at the i-th codimension-2 
face). Using the product rule, we see that 



When the deformation is isometric, the first sum on the right hand side vanishes, and, combining 
that with the formula 1, we see that 
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where the minus sign comes from replacing the dihedral angles with exterior dihedral angles. 

When K = 0, the right-hand side is 0. This shows that the "mean curvature" of a 1-parameter 
family of Euclidean polyhedra with constant induced metric is constant. This has been used in 
[AR97] to prove, using geometric measure theory methods, a similar result for isometric deforma- 
tions of smooth hypersurfaces in space- forms; we prove this result directly by differential-geometric 
methods here (in the more general setting of Einstein manifolds), indeed, we show the rather strong 
Theorem ^ Theorem ^ should be compared with the celebrated mean curvature variation (under 
bending) formula of Herglotz; see | BG93 |. 

Isometr ic defo rmations of (smo o th) hyp ersurfaces remain rather mysterious. It is known since 
Liebmann jLieOCtl (also see [|Her79|] , jSpirsI , vol V]) that (strictly) convex surfaces in R'^ admit no 



such deformation, and this has been extended to and by Pogorelov [Pog73 . On the other 



hand, it is unknown whether any smooth closed surface in R'^, or has a 1-parameter family 
of isometric deformations, or even if any closed surface with no open flat region has a smooth 
infinitesimal isometric deformation. But it is known that there exist non-rigid polyhedra, see 
| Con77| and iBle96| . 

A smooth version of (|lj) leads to some rigidity results. Aside from the remarks following 
Theorem 1^, we prove in Section 3 a rigidity result for Ricci-fiat manifolds with umbilic boundaries, 
with respect to Einstein deformations of the metric which leave the induced metric on the boundary 
fixed. 

Section 4 contains some applications of our "Schlafli formula" to codimension one foliations 
of Einstein manifolds. Section 5 recalls some concepts of integral geometry, to see how some of 
our results may be recast in integral-geometric terms. In This is particularly relevant to section 6 
(after giving brief preliminaries in Section 5), where we extend a classical inequality of Alexandrov 
for convex bodies in Euclidean space to the hyperbolic and spherical settings. 

This paper can be considered (among other things) as a hint that some important elements of 
hyperbolic geometry in dimension 3 can be extended in higher dim ension in the setting of Einstein 
manifolds (with negative curvature). Another such hint is given in [ 3ch98 |, which contains a partial 
extension of some classical results of the theory of convex surfaces in hyperbolic 3-space to Einstein 
manifolds with boundary. 

The smoo th Sch lafli formula given in section 1 also has analogous "higher" smooth Schlafli 
formulas (sec | SS99 |) but in constant curvature manifolds only. Going to the polyhedral case leads 



2 



to "higher" polyhedral Schlafli formulas, relating the variations of the volumes of the p-faces to 
the variations of the "curvatures" of the {p + 2)-faces. Similar (polyhedral) formulas were given 



(in some special cases) in [3P97| 



Throughout this paper, M is an Einstein manifold of dimension m + 1 > 3, and D is its Levi- 
Civita connection. When dealing with a hypersurface S (resp. with the boundary dM), we call 
I the induced metric, also called the first fundamental form, of the corresponding immersion in 
M. D is the Levi-Civita connection of /, and B the shape operator defined, for any a; G S (resp. 
X G dM) and X G T^^ (resp. X G T.^.dM) by 

BX = -Dxn , 

where n is the oriented normal unit vector to E (resp. the unit exterior normal to dM). The 
second fundamental form E oiT, (resp. dM) is defined by 

n{X,Y)^I{X,BY) , 

and the third fundamental form by 

M{X,Y) = I{BX,BY) . 

The trace iJ of B is called the "mean curvature" (some definitions differ by a factor m) and the 
"higher mean curvatures" H}^ , A: > 1 , are the higher symmetric functions of the principal curvatures 
of E (resp. dM). For instance, H2 ~ {H^ — tr(B^))/2. dV,dA are the volume elements in M and 
on E (resp. dM) respectively. 

We denote the divergence acting on symmetric tensors by S, and its formal adjoint by 6*. 
Therefore, if ft, is a symmetric 2-tensor and (ei)igN„,+i an orthonormal moving frame on M, then, 
for any vector x G TM: 

(^/i)(x) --^(i?e.M(e^,a.) , 

i 

and, if 1? is a vector field, p G M, a:, y G TpM, then: 

{S*v){x,y)^^{{D,v,y) + {DyV,x)) . 

We will often implicitly identify (through the metric) vector fields and 1-forms, as well as 
quadratic forms and linear morphisms. 



1 Deformation of hypersurfaces 

This section contains an analogue of the Schlafli formula for deformations of (smooth) hypersur- 
faces in a fixed Einstein manifold M, which can be Riemannian or Lorentzian (the other pseudo- 
Riemannian cases can be treated in the same way; we have not included them to keep things as 
simple as possible) . This contrasts with the results in the next section, where the same formula is 
proved for variations of the metric inside a manifold with boundary (which is much more general) 
but only when M is Riemannian. 

We also show how this "smooth" Schlafli formula can be used to recover the classical polyhedral 
formula (|l|), in the Riemannian and Lorentzian cases. 

The techniques here are quite elementary, and use the method of moving frames. 

Theorem 1 Let E 6e a smooth oriented hypersurface in a (Riemannian) Einstein {m+ 1) -manifold 
M with scalar curvature S , and v a section of the restriction ofTM to T,. v defines a deformation 
of E in M, which induces variations V',H' and I' of the volume bounded by E, mean curvature, 
and induced metric on E. Then: 

^ / H' + hl',F)dA (2) 
m+l 2 
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S actually does not need bound a finite volume domain for this formula to hold. If it doesn't, 
then V doesn't exist, but its variation still makes sense (since S is homologous to its images under 

deformations). 

Proof: We first prove the formula for v tangent to S, then we'll check for normal vector fields. 
When V is tangent to S, V = 0, and 

I'{X,Y) = {Dxv,Y) + {Dyv,X) = 2{6*v){X,Y) 

so that: 

/ {I',F)dA = 2 I {6*v,E)dA = 2 f {v,5E)dA 
i/ XI I' s ■/ yi 

Let (ej) be an orthonormal frame for / for which B is diagonal. The Codazzi equation shows that 

{DxE){Y, Z) = (DyJI){X, Z) + {R{X, Y)n, Z) 

so 

= -{DyE){ei,ei) - {R{ei, v)n, e,) 
= —dH{v) + ric(v, n) 

Now M is Einstein and n is orthogonal to v, so that: 

{5E,v) = -dH{v) (3) 

Therefore: 

/ {I',E)dA = -2 [ dH{v)dA 

This proves the formula when v is tangent to S. 

Suppose now that i; is a normal vector field, i.e. v = fn for some function / on S. Since / is 
the difference between two strictly positive functions, it is enough to prove the result when / does 
not vanish. Let x, y be vector fields on E. Choose an extension of fn to some vector field on a 
neighborhood Q of S in M, with n the unit orthogonal to the image of S by the flow of fn, and 
df{n) = 0. Extend x,y to Q by the flow of fn, then [fn, x] = [fn, y] = 0. We now have: 

I'{x,y) = fn.{x,y) = {DfnX,y) + {x,Djny) = {D^{fn),y) + {x,Dy{fn)) = 2fE{x,y) 

so /' = —2fE. One also checks that Dfnn = —Df, so that: 

E'{x,y) = -fn.{Dxn,y) 

= -{DfnDxn, y) - {D^n, Df^y) 

= -{D^Dfnn + Rfn,xn + Dyn,x\n, y) - {D^n, Dy{fn)) 
= {D^Df, y) - {Rfn,xn, y) - fEI{x, y) 

and 

E' = Hf-f{Rr,,n,-)-fm (4) 

where Hf is the Hessian of / on S. 
Taking the trace of this equation: 

H' = tr{E') - (/', J) = -A/ + /ric(n, n) - /tr(iff) - (/', E) 

Now the integral over S of A/ is zero, and the integral of / is V because the deformation is 
normal. The result follows, because 7' = -2fE, so that -2/tr(iff) = {I',E). □ 

This formula leads easily to the "classical" Schlafli formula for polyhedra in space-forms: 
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Theorem 2 Let P be a convex polyhedron in a [m + 1)- dimensional space-form M with scalar 
curvature S ; for any deformation of P , the variation V' of the volume bounded by P is given in 
term of the variations 9[ of the dihedral angles at the codimension 2 faces by: 

m + l ^ 

where Wi is the (m — 1) -volume of the codimension 2 face i. 

Proof: First note that Theorem |^ also apphes for deformatfons of a C^'^, piecewise smooth 
hypersurface (if the deformation preserves the decomposition into smooth parts). This is proved 
by an easy approximation argument. The formula remains the same, and each term make sense in 
this case. 

Call Pe the set of points at distance e of P on the outside (i.e. on the side of P which is concave). 
For e small enough, Pf is a C^'^, piecewise smooth hypersurface, and we can apply Theorem |l|. 
Note Je, H'^, VI the quantities corresponding to /', E, H' , V for P^. Then: 

S f „, 1 



= H' + \{I[,E,)dA 



For e small enough, we can decompose P^ as 

p _ , ,m+l p 

where P^^ k is the set of points where the normal meets P on a codimension k face. Using the flow 
of the unit normal vectors to the Pe, we can also identify P^ and Pe' for e' ^ e, so that we can 
consider e.g. as a 1-parameter family of symmetric 2-tensors on a fixed manifold. 

If x G Pe,2, then the normal to Pe at x meets some codimension 2 face Fi of P; let ai^t be the 
dihedral angle at Fi. li v^w ^ T^Pe correspond to vectors orthogonal to TF, then 

I',iv,w) ~ —^^^h{v,w) 
ai^t at 

as e ^ 0. On the other hand, 

Seiv^w) ~ -/e(u, w) 

li v,w now correspond to vectors in TFi, then 

l'^iv,w)^0{l) , 

while 

E,{v,w) = 0{e) . 
Using those 2 cases, we see that, at any point in Pe^2: 

£cti,t dt 

Now the volume element of Pe,2 is equivalent to e as e — > 0, so: 

For Pe^i (that is, for codimension 1 faces), only vectors parallel to the faces have to be taken 
into account, and their contribution is of order 0{e) (as above for Pe,2)- 
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For Pe.fc with k > 3, the same reasoning shows that only vectors orthogonal to the faces count; 
a v,w are such vectors, then 

I'^{v,w)^Oih{v,w)) , 

while 

E,iv,w) = 0{hiv,w)/e) , 
and the volume element on P^^k is as 0{e'^^^), so 

lim / (I'FAdA = . 

It is also easy to check that 



hm / H'dA^Q , 



and this leads to the formula. □ 



Of course, P does not need to be convex: once the corollary is proved for convex polyhedra, it 
is clear that it also applies to non-convex ones, since they can be decomposed into convex pieces. 

The proof of Theorem |^ also applies to the Lorentzian case. The only difference is that now 
g{n, n) — —1, so the volume variation has a minus sign in the formula. 

Theorem 3 Let Yi he a smooth oriented space-like hypersurface in a Lorentzian Einstein (m + 1)- 
manifold {M,g) with ricg = mkg, and let v be a section of the restriction ofTM to E. v defines a 
deformation ofTi in M , which induces variations V , H' and I' of the volume bounded by S, mean 
curvature, and induced metric on E. Then: 

mkV = [ H' + ]-{L' ,E)dA (5) 

Here again, the volume might be defined only up to an additive constant (for instance as 
the volume bounded by S and some fixed homologous hypersurface Eg), but its variation is well 
defined. For instance, if E is a compact space-like hypersurface in the de Sitter space, its "volume" 
can be defined as the oriented volume of the domain bounded by E and by some space-like totally 
geodesic hyperplane 5*0. This volume actually does not depend on So, because if 5*1 is some other 
totally geodesic hyperplane, then, as (5) shows, the oriented volume of the domain bounded by Sq 
and 5'i is zero. 

This lemma could actually be extended almost without change to other pseudo-Riemannian 
manifolds, and also to hypersurfaces which are not space-like. 

Applying this lemma to the set of points at distance e from a polyhedron in S*" (as above in 
Theorem 2), one obtains the Schlafii formula for de Sitter polyhedra as in pP97[ | (where it was 
proved for simplices using a more combinatorial approach). 

Theorem 4 Let P be a convex space-like polyhedron in the de Sitter space 5™^^, which is dual to 
a hyperbolic polyhedron. For any deformation of P , the variation V' of the volume bounded by P 
is given in term of the variations 9[ of the dihedral angles at the codimension 2 faces by: 

mV + w.,e[ = 

i 

where Wi is the (m — l)-volume of the codimension 2 face i. 

The conditions that P is convex and dual to a hyperbolic polyhedron are actually not necessary, 
and the formula even remains valid for many polyhedra that are not space-like. It then helps to 
use a definition of angles and volume well adapted to this Lorentzian setting, i.e. with complex 



6 



values (as in |Sch|). It is not obvious how to give a complete proof using smooth formulas (as 
above) but many cases can be treated simply by using sums or differences of polyhedra for which 
smooth formulas work. For instance, this can be done for all space-like polyhedra. 
Theorem ^ applied in Euclidean space leads to the 

Theorem 5 In R'"+^, the integral of the mean curvature remains constant under an isometric 
deformation of a compact hypersurface. 

On the other hand, the integral mean curvature is not determined by the metric on dM: this 
is already visible in R"^. Namely, some metrics on S'^ admit two isometric embeddings in R'^: the 
classical example is that a (topological) sphere in R^ which is tangent to a plane along a circle 
can be "flipped" so as to obtain another embedding with the same induced metric | 3pi75 |. Those 
two embeddings do not in general have the same integral mean curvature - and thus we have a 
complicated way of seeing that the two flipped surfaces cannot be bent one into the other. 

The analogue of Theorem]^ is also true, but in a pointwise sense, for the higher mean curvatures: 

Theorem 6 In R™+i, the integral of Hk (k > 2) remains constant in an isometric deformation 
of a hypersurface. 

This comes from the following (probably classical) description of the possible isometric defor- 
mations of a hypersurface for m + 1 > A: 

Remark 1 Let {'Et)t£[o,i] be a 1 -parameter family of hypersurfaces in a space-form, such that the 
induced metric It is constant to the first order at t = 0. Then, at each point, one of the following 
is true: 

• Ha = 0; 

• rfc(iro) < 2, and Eq vanishes on the kernel of Eq; 
. = 0. 

where Et is the second fundamental form ofT.t, and E^ its variation. 

Theorem ^ clearly follows, because Hj^ is zero for fc > 3 in each case, and the Gauss formula 
gives the proof for k — 2. 

Proof of Remark |l|: Choose an orthonormal frame (ei , • • • , e™) on So for which Jo is diagonal, 
with eigenvalues (fci, • • • , /cm). By the Gauss formula, Et @ Et (where Q is the Kulkarni-Nomizu 
product) is determined by the induced metric, and is thus independent on t. Therefore, for any 
choice of indices p, q,r,s: 

Eo{ep,es)Eg{eq,er) -\- E^icp, es)EQ{eq, er) = Eo{ep,er)El){eg,es) -|- -ffo(ep, 6^)^0(6,, e^) 

Taking p, q, r distinct but s — p shows that 

fcpJ^(e„e,) = (6) 

while taking p = s ^ q = r leads to: 

kpEQ{eq,eq) + kqEgiep^ep) ^ (7) 

Consider the case where rk(iro) > 3. For each choice of p, q, r with fcp, kq, kr ^ 0, adding eq. (7) 
(divided by kpkq) for the pairs {p,q) and {p,r) and subtracting the same equation for the pair 
[q, r) shows that E^^Cp, Cp) = 0, and the same for q, r, so we already see that all diagonal terms of 
Eq are zero. Then eq. (||) shows that all non-diagonal terms are zero too, so Eq = 0. 
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If rk(iro) < 2 but Mq ^ 0, then eq. and eq. (7) easily show that Mq = except maybe in the 
subspace generated by the eigenvectors of Eq with non-zero eigenvalue. □ 

Theorems || and || can be combined to give the following geometric statement. Denote by the 
parallel surface at distance e from Sj. It is well-known (see, eg, Santalo's book |3an76|) that the 
area of E'^ is a polynomial in e where the coefficient of e'^ is (essentially) the k-th mean curvature 
of S. The two Theorems |] and ^ can than be combined as stating that: 

Theorem 7 The area ofY,\ stays constant when Et is a bending ofTiQ. 



2 Einstein manifolds with boundary 

In this section, (M, dM) is a compact manifold with boundary with an Einstein metric g of scalar 
curvature S. We will prove the same formula as in the previous section, but in a much more general 
setting: instead of moving a hypersurface in an Einstein manifold, we will be changing the metric 
(among Einstein metrics of given scalar curvature) inside this manifold with boundary. Although 
the two operations are equivalent in dimension at most 3, moving the inside metric is much more 
general in higher dimension. On the other hand, our proof only works for Riemannian Einstein 
manifolds. It is not obvious whether it can be extended to the pseudo- riemannian setting. 

As always when studying deformations of Riemannian metrics, we need put some kind of restric- 
tion to remove the indeterminacy coming from the fact that some deformations are geometrically 
trivial, that is, they just correspond to the action of vector fields on the metric. We prevent those 



deformations in the same way as e.g. in |GL91], |DeT81| or [Biq97|, that is, we only consider 



metric variations h such that 25h + dtTh = 0. The following proposition shows that we don't forget 
any metric variation when doing this. 

Proposition 1 Let h' be a smooth variation of g. Suppose that either S < 0, or that M is 
strictly convex. There exists another smooth variation h of g such that 2Sh + dtr(h) = and that 
h = h' + S*Vo, where vq is a vector field vanishing on dM . 

Proof: Suppose t; is a vector field on A/, let h — h' + S*v. Then 

25h + dtrh = 2Sh' + dtrh' + 25{5*v) + dtr{S*v) . 

Now, if a; is a vector field on M: 

26{6*v){x) = -Y,2{D,^{S*v)){e^,x) 

i 

= Y.~e,.{26*v){e,,x) + {26*v){D,^e,,x) + {25*v){e^,D,^x) , 

i 

SO 

2S{6*v){x) = J2 -^U{De,v, x) + {D^v, e,))-f 

i 

+ {DD^^eiV, x) + {D^V, Deiei) + {DeiV, DeiX) + {DD,^xV, Ci) 
25{5*V){X) = "^{-De,De,V + Do^^e.V^x) + {^De,DxV + Do^^xV^ei) . 



and 



On the other hand: 



d{tTi6*v))ix) = X. (^{D,^v,e,)^ = J2(Dx 



De^V^Ci) + {De,V,Dxei) , 
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so that 

{2S{S*v)+dtr{S*v))ix) = ^( D^^D^^v+DD,^e,v,x)-{Re„xV,e^) + {DD,e,v,e,) + {De^v,D^e^) . 

i 

If LU is the connection form of the frame (e7)teN„^+i5 then 

i i 

because 5*v is symmetric and uj{x) is skew-symmetric. Therefore, 

{25{5*v)+diT{5*v)){x) = {D*Dv,x) -Tic{v,x) = {D*Dv,x) ^{v,x) , 

m + 1 

so 

2S{S*v)+dtr{S*v) ^ D*Dv —v. (8) 

m + 1 



To prove the proposition, we have to solve the elhptic problem: 

_s_ 

n+ 



D*Dv-:^v = -{26h' + dtr{h')) 



(9) 



Call FqTA/ the space of vector fields on M which are in the Sobolev space and whose trace 
on dM vanishes (this essentially means that they are zero on dM), and define 



F : TlTM R 

V ^ y,.^{Dv,Dv)-^^{v,v)dV + j\,j{2Sh' + dtrih'),v)dV. 

Then F is strictly convex, and moreover it is coercive; this is clear if 5 < 0, and, if 5 = 0, it follows 
from the Poincare inequality for vector fields vanishing on dM : 

3CyveTlTM,[ {Dv,Dv)dV>c[ {v,v)dV 

If 5 > 0, a more careful argument is necessary. Let u := \\v\\. Then 

{Du,Du) < {Dv,Dv) 

and 

/ {Du,Du)dV > Xi I u^dV , 
Jm Jm 

where Ai is the first eigenvalue of the Dirichlet problem for the Laplacian on AI. But it is known 
(see [Rei77], |Kas84 |) that, for M convex, and under the hypothesis that the Ricci curvature is 
bounded below by S/ (m + 1) 

Ai > - , 
m 

with equality if and only if AI is a hemisphere. Therefore, 

S f 

(Dv,Dv)dV >— {v,v)dV 
M ^ Jm 

and F is again coercive. 

Therefore, F admits a unique minimum vq on TqTM, which is smooth by standard elliptic 
arguments. Then, for all u G TTM, 

{n„F)iu) = 
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so that 

/ (Dvo,Du) —{vo,u)dV+ I {26h' + dtr{h'),u)dV = 

Jm fn+1 Jm 



I (D*Dvo ^vo + 26h' + dtr(h'),u)dV = 

Jm m+l 



I M 

and D*Dvo - :^^vo = -2Sh' - dtv{h') as needed. □ 

Another way of solving eq. (9) would be to check that it has index 0, and that if 26{S*v) + 
dtr{6*v) = on M and w = on dM, then v = 0. 

If g is an Einstein metric, we say that a 2-tensor h is an "Einstein variation" of 5 if the associated 
variation of the metric induces a variation of the Ricci tensor which is proportional to h, so that 
g + eh remains, to the first order, an Einstein manifold with constant scalar curvature. 



Theorem 8 Let h be a smooth Einstein variation of g. Then: 

S 



m + 



-V'= [ H' + \{h\0M,a)dA 

^ JdM ^ 



(10) 



Proof: By the previous proposition, wc can suppose that 2Sh + dtT{h) = 0. First, wc compute 
the variation E' of E on dM. Let a; be a vector field on M so that D^x = 0. Extend n to a unit 
vector field on M such that -D„n = 0. Then 

2E{x,x) = —2{Dxn,x) = —n.{x,x) 2{[x,n],x) . 

Now, since n remains the unit normal to dM 

n = ~ 2^ ~ 

where r = h{n, n) and a G TdM is such that for any vector y e TdM, {y, a) = h{n, y). Therefore, 

2E'{x,x) = —n.h{x,x) — 2h{[x,n],x)+(^^n + a^.{x,x) + {[x,Tn + 2a],x) 

= —{Dnh){x,x) + 2h{Bx,x) + a.{x,x) — t{Bx,x) + 2{[x,a],x) 
= -{Dnh){x, x) + 2h{Bx, x) + 2(£)^a, x) - t{Bx, x) . 

To go further, we note 6 the divergence on dM, a the 1-form dual to a on dM, and t := tr(/i). 

If («!,•••, Un) is an orthonormal frame on dM for which E is diagonal, extended on M so that 
DnUi = 0, we have: 

2tr(J') = - Ui) + 2{h, E) 2(5a) - Ttr(ir) . 



But 



^(-D„/i)(ui, Ui) = -dt{n) + {Dnh){n, n) 

i 

= -dt{n) - {5h){n) - Y,{DuMui, n) 

i 

= -^{D^Mui,n) 

i 

— ^^^^^ ^Ui.a{ui) + h{DuiUi,n) + hiui.Du^n) 

i 

2 - ^{Duia){ui) + E{ui,Ui)T - h{ui, Bm) 



2 

dt{n) 
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and, finally, 

2trin') = -^ + {h,E)-iSa). (11) 

The statement that h is an Einstein variation of g can be written (see [Bes87], chapter 12) as the 
equation: 

D*Dh - 2Rh - 2S*Sh - Ddt = , 
where R is the curvature operator acting on symmetric 2-tensors; and, since 2dh + dt = 0: 

D*Dh - 2Rh = . 

Taking the trace of this equation, we find that: 

2S 



An elementary computation shows that the variation of the volume of M is equal to half the 
integral of the trace t of h: 

2V' = / tdV . 



M 



But 

25 



m + 1 

and, using eq. (pj), we obtain: 



/ tdV = AtdV = - dt{n)dA 

Jm Jm JdM 



2 / H'dA^ / 2ti{E')-2{h\QM,II)dA 

JdM JdM 



^ + {h^9M,]I) + iSa)dA = -^V- I {h\9M,]I)dA 
dM ^ m + 1 Jqm 



from which the result follows. □ 



Formula (10) is even simpler for variations which vanish on dM: 

Theorem 9 If h is a smooth Einstein variation of g which does not change the induced metric on 
DM, then 

H'dV - — 

dM m+l 

In particular, for 5 = 0, this implies that the integral of the mean curvature of the boundary 
is constant under an Einstein variation which does not change the induced metric on dM] this is 
a direct generalization of Theorem |^. 

A more interesting application can be found by looking at "singular objects" , just as we did 
to get the polyhedral Theorem 2 from the smooth Lemma |l]. There are no polyhedra in general 
Einstein manifolds, but we can check what happens when we deform Einstein manifolds with cone 
singularities. It should be pointed out that some of the most interesting modern uses of the classical 
Schlafli formula concern hyperbolic 3-dimensional cone- manifolds. 

Let M be a compact (m + l)-manifold, and N a compact codimension 2 submanifold of Ad . 
Suppose {gt) is a 1-parameter family of Einstein metrics with fixed scalar curvature 5 < on 
M \ N , with a conical singularity on TV in the sense that, in normal coordinates around iV, g^ has 
an expansion like: 

gt = ht+dr^ +r^d0^ +o{r^) 

where ht is the metric induced on N by gt, and 6 £ R/afZ for some at G R. Call Vt the volume 
of (M \ N,gt), and Wt the volume of (iV, ht). Then: 
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Corollary 1 Vt varies as follows: 



S dVt dat 
Wt- 



m + 1 dt dt 

The same formula of course remains true if N has several connected components, each with a 
different value of at ■ 

Proof: It is similar to the proof of Theorem 2, so we go a little faster. Set 

N^{t) ^ [xe M , d{x, N) > eat} 

Apply Theorem ^ to the boundary of Ne{t) and take the limit as e — ^ 0. Again, we consider dN^ 
as a fixed manifold (diffeomorphic to x S*^) with a 1-parameter family of metrics depending on 
e. li v,w ^ TdNf: correspond to vectors in TN, then 

l'Jv,w)^0{l) 

while 

E,(v,w)=0{e) 
If w G TdN^ corresponds to a vector normal to A^, then 

T'l \ dat , ^ 
at dt 

while 

Ue{v,v) ~ 

so that finally 

eat dt 

On the other hand, the mean curvature of the boundary (given only by the d/d9 direction) is 
and 

H'.=o(l 

so that we finally find that 

S 



m 

a 



-F/ = lim / l{l'^,JI,)dA=^Wt 
1 ^^0J97V,(t) 2 dt 



Note: The same computation could be made with A^ replaced by a stratified submanifold; the 
same result follows. 

Example: take m + 1 = 3 in the previous example. We find the Schlafli formula for the 
variation of the volume of a hyperbolic cone-manifold. 
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3 Applications to rigidity 



In this section, we use the Schlafli formula above to prove a rigidity result for Ri cci-flat manifolds 
with umbilic boundary; it is a generalization of the classical result (see [ Spi75|l ) that the round 
sphere is rigid in R'^, that is, it can not be deformed smoothly without changing its induced metric. 

This kind of rigidity result could be used in the future to prove that, given a Ricci-flat manifold 
M with umbilic boundary and induced metric go on the boundary, any metric close to go on 
dM can be realized as induced on dM by some Ricci-flat metric on M. In this setting, rigidity 
corresponds to the local injectivity of an operator sending the metrics on M to the metrics on dM . 



In dimension 3, this would be a part of the classical result (see |Nir53 ) that metrics with curvature 



K > on S can be realized as induced by immersions into R''. This circle of ideas is illustrated 



in [3ch98|. It is rather remarkable that the same condition (that the boundary is umbilic) appears 



both here and in |Sch98|, in the same kind of rigidity questions, but in a very different way. 

The first point is to understand what an umbilic hypersurface in an Einstein manifold is. By 
definition, if iV is a Riemannian manifold and is 5* a hypersurface, then S is umbilic if, at each 
point s £ S, E is proportional to /, with a proportionality constant A(s) depending on s. Now 

Remark 2 If N is Einstein, then A is constant on each connected component of S . 

Proof: Let B be the shape operator of S, and n the unit normal. For s £ S and x,y £ T^S, 
the Codazzi formula asserts that: _ 

{dPB){x, y) = Rx,yn 

where D is the Levi-Civita connection of 5, and R the curvature operator of N . Since E = A/, 
this means that: 

{d\{x))y - {dX{y))x = Rx,yn 

Let (ei)i<i<„i be a moving frame on S. Taking the trace of the previous expression with respect 
to y shows that: 



idX{x) — dX{ei){x, i 



= ric^a;, n) 

i=l 

and ric(a:, n) = because N is Einstein and n is orthogonal to x. Therefore: 

(m - l)<iA(a;) = 

for any tangent vector s to S, so A is locally constant. □ 

Corollary 2 Umbilic hypersurfaces of Einstein manifolds are analytic. 

A rather clumsy way to prove this is to note that, because of the previous remark, umbilic 
hypersurfaces are locally graphs of solutions of some elliptic PDE with analytic coefficients (because 
Einstein metrics are analytic, see | Bcs87| ). A classical elliptic smoothness theorem then gives the 



result. As a consequence: 

Corollary 3 Let (E, h) be a compact analytic Riemannian m-manifold, and X, S £ R. There 
exists at most one germ of Einstein (m + l)-manifold around S with scalar curvature S which 
induces h and for which E is umbilic with E = XI . 

Proof: Let M be such a germ of Einstein manifold around E, and g its metric. By taking the 
geodesic flow of the exponential normal to E in M, we see that g can be locally written, in some 
neighborhood 1^ of E, as 

g = kt + dt^ . 
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We call Et the second fundamental form of the hypersurface E x {t} for g, and Mt the corresponding 
third fundamental form. Choose m g E, and a;,y G TmYi- Then a classical computation (which 
was done, in a slightly more general case, in the proof of Lemma |l|) shows that: 

^ (a;, y) = -Hit {x, y) + {R{x, n)y, n) . 

Let (ci) be an orthonormal frame at m. Call R the Riemann curvature tensor of g, and Rt the 
curvature tensor of fcf. By the Gauss formula, for i £ N: 

{Rt{x,ei)y,ei) = {R{x, ei)y, Ci) + {Et ®Et){x,ei,y,ei) , 

where © is the Kulkarni-Nomizu product (see | Bes87] ] ) . Taking the trace and calling ric the Ricci 
curvature of M and ricj the Ricci curvature of fcj leads to: 

nct{x,y) = ric{x,y) - {R{x,n)y,n) +^(Jf @Ft){x,ei,y,ei) 



and 



so that 



J2{]It ®Ilt){x,e^,y,ei) = HtFt{x,y) - mt{x,y) 



i=l 



— ^ = ric - rict + HtEt~ 2EIt 
at 



Now Et = —dkt/dt, and rict can be considered as a second-order elliptic operator in fcj. So kt 
satisfies 



dt^ -^(^*)+^Uj 

where P is a second-order elliptic operator and R is an operator of degree (all solutions of eq. (12) 
do not correspond to germs of Einstein manifolds around an umbilic surface; for instance, for 5 = 
and TO = 2, only constant curvature metrics can be induced on umbilic surfaces in Euclidean space). 

Now we can apply the Cauchy-Kowalevskaya theorem (or the Cartan-Kahler theorem) which 
shows that eq. (|l^) has a unique analytic solution. On the other hand, Corollary || asserts that E 
has to be analytic in M, and then kt has to be analytic, and also g (see [Bes87], 5.F). Equation 
( p^ ) therefore has a unique solution, which is analytic. □ 



Note that the solutions of equation (y_2|) might not correspond to a germ of Einstein manifold 
around E, but such a germ is always obtained as a solution of eq. (p^, and so is unique. 
The next step is an inequality concerning the integral of the mean curvature squared. 

Proposition 2 Let {M,g) be an Einstein manifold with boundary, with scalar curvature S. Call 
S the scalar curvature of (dM, g^gm)- Then the mean curvature H = tr{E) of dM satisfies 

S S 

< 



TO — 1 TO -f 1 TO 

with equality if and only if dM is umbilic. 

Proof: Call ric the Ricci curvature of {dM, g^gj^j), and K{x,y) (resp. K{x,y)) the sectional 
curvature of g (resp. g\gM) on the 2-plane generated by x,y. Let (ui, • ■ • ,m„) be an orthonormal 
frame on dM for which E is diagonal, with eigenvalues fci, • • • , /c„. Then, by the Gauss formula: 

K{ui,Uj) = K{ui,Uj) + kikj 
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— 29 

S = S- 2ric(n, n) + V k,(H - k,) + -J^ kf . (13) 



and, taking the trace: 

ric(ui, Ui) = ric(ui, Ui) - K{ui, n) + k 
Taking the trace once more: 

m + 1 
Now 

with equahty if and only if all ki are equal. The result follows. □ 

This inequality could be interesting in itself. For instance, if 5 = —m{m + 1) and H is bounded 
above by some constant, it implies that the scalar curvature of dM is negative, which has some 
topological consequences. 

By the way, this computation also leads to the following partial extension of Theorem 6: 

Remark 3 The second mean curvature H2 — X]i<j ^i^j '^f boundary of an Einstein manifold 
is: 

m + 1 

Therefore, it is pointwise constant in an Einstein variation of the metric which vanishes on the 
boundary. 



Proof: It follows from eq. ([13[). □ 

Now from Proposition]^ and Lemma ^ we get: 

Corollary 4 Let {M,dM) be a compact manifold with convex (or concave) boundary. Let 
{gt)t^[o,i] be a one-parameter family of Einstein metrics on M with scalar curvature S, such that 
dM is umbilic for go, and that the metric induced by gt on dM is constant. Call Tit the integral 
of the mean curvature of H for gt . Then: 

1. if S > and H > (resp. H < 0) on each connected component of dM, then both Tit and 
Vt are minimal (resp. maximal) for to, and the variation H[ of Ht vanishes for t — 0; 

2. if S < and H > (resp. H < 0) on each connected component of dM, then Tit is minimal 
and Vt is maximal (resp. Tit is maximal and Vt is minimal) for to, and the variation Ht of 
Ht vanishes for t = 0; 

3. if S — 0, then dM is umbilic for all t G [0, 1], i.e. its second fundamental form does not 
change. 

Proof: By Proposition |^, iJ^ is pointwise minimal over dM when dM is umbilic. If, for 
instance, > on each connected component of dM , this shows that Tit is also minimal for i = 0. 
By Corollary ^, V is also minimal for t = when > 0, and maximal when 5 < 0. This proves 
assertions 1 and 2. 

For assertion 3, the integral of H over dM is constant by Corollary ^, while H^ is pointwise 
minimal over dM . Therefore, H^ has to be constant. By the equality case in Proposition ||, dM 
has to remain umbilic in that case. □ 

The Corollaries || and ^ lead to a description of the non-rigid Ricci-flat manifolds with umbilic 
boundary. 
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Lemma 1 Suppose (M, DM) is a compact {m+1) -manifold with boundary, and iht)te[o,i] non- 
trivial 1-parameter family of Ricci-flat metrics on M inducing the same metric on dM, and such 
that dM is umbilic for ho- Then dM has at least 2 connected components, and (ht) corresponds 
to the displacement of some connected component(s) of dM under the flow of some Killing field(s) 
ofM. 

Note that this is rather restrictive, since a "generic" Einstein manifold with boundary should 
not admit any Killing field. Some examples are given bellow. 

Proof: By Corollary ^, h does not change the induced metric or the second fundamental form 
of dM. Call El, • ■ • , Ejv the connected components of dM. Then, by Corollary ^, each connected 
component diM of dM has a neighborhood Vli^t which does not change. Therefore, the deformation 
{ht) corresponds to the displacement of some diM under Killing fields on A/. □ 

We shall give some examples of what can happen; this is easier using the following elementary: 

Proposition 3 Let {N^go) be an Einstein m-manifold with scalar curvature m{m — l)fc, consider 
the product M x R with the warped metric 

g -.^ dt^ + fitfgo 

where f is a function defined on some interval / C R. Then g is Einstein with scalar curvature 
m(m+l)fc' if and only if f"{t) = —k' fit) and k — k' f (t)^ -\- f (t)'^ for allt. Then each hypersurface 
N X {t} is umbilic in M . 

Proof: First check that the Levi-Civita connection D of g is related to the Levi-Civita con- 
nection D of 50 by the following formulas: if n is the unit normal vector to TV x {t\, and x,y are 
vector fields on N (and their extensions on M) then 

— /' 

Dxy = D^y - —g{x,y)n 

/' 

Dj^x = D^n = —X 

DnU = 

This is because those expressions define a torsion-free connection compatible with g. 

Now this expressions of D shows that each hypersurface N x {to} in M is umbilic, with second 
fundamental form —{f'/f)g. Therefore, the Gauss formula shows that the sectional curvature of 
M on any 2-plane tangent to x {to} is: 

1 

Km — j2^^ ^ ~p 
and on each 2-plane containing the direction normal to iV x {to}: 

which shows that ric*^(n,n) = —rnf"/f, and leads to the first condition. Taking a trace, we see 
that for X tangent to x {t}: 

ric'" {x,x) = 9{x,x) 

and the second condition follows. □ 

Since we are interested in Ricci-flat metrics, we have to take k' = 0, and we can use this 
proposition in two ways: either k = Q and /(t) = 1, or fc = 1 and /(t) — t. 

The simplest example is the case when go is the canonical metric on the sphere 5"*: 
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Example 1 Consider the unit ball c R™^^. Any 1-parameter Einstein deformation of the 

metric in 5'"+^ which doesn't change the induced metric on the boundary 5*™ is trivial. 

On the other hand: 

Example 2 Consider the "cylinder" Q :— x [0,1]. There exists a I -parameter family of 
deformations of the metric on Q which does not change the induced metric on the boundary. 

This deformation is obtained by one of the boundary components along the axis of the cyhnder. 
This happens because there is a KiUing vector field, which is parallel to the axis. 



4 Codimension one foliations 

We give in this section some simple formulas obtained by applying Theorem ^ to codimension one 
foliations of Einstein manifolds. Let {'Et)tei be a smooth one-parameter family of hypersurfaces 
in an Einstein (rri + l)-manifold with scalar curvature m{m + 1)K . Suppose that the define a 
foliation of a domain ^l C M. For each x G il, let H2{x) be the second mean curvature of at x 
(for t such that a; G St) and let Sy,{x) be the scalar curvature of Et for the induced metric. Then: 

Theorem 10 The volume V{^) of 57 is: 

mKV{n) = 2 / H2dV + [ H{dVl)dA , (14) 
Jn Jan 

mKV{n)= [ Hi - tr{mT.)dV + [ H[dn)dA , (15) 
Jii. Jan 

m^KV{n) ^ [ SY.dV + [ H[dn)dA . (16) 
Jn Jdn 



and also 



Here H{dfl) is the mean curvature of dfl. H^, Sy: and are the mean curvature, sectional 
curvature and third fundamental form of St . 

Proof: Suppose for instance that / = [0, 1], and note: 

Ft = Vol (uUoSs) . 

Denote again the variations of Vt,/t and Ht by a prime. Choose a parametrization (pt : T, ^ M 
such that St = (/'t(S), and let (f)[ = v + fn, where v e TS and n is the unit normal vector to St. 
Then, by eq. (2): 



nKV; = j H', + hl[,Et)dA 

■ I HtdA- I HtdA'+ I Et)dA 



dt 
d 
di 



d r , . r 1 



[ HtdA + ( Jt) - Ht{-fHt + dxY{v))dA 

= / HtdA + f -f{Et,at) + {Tv, Ft) + fHf - Ht'^{v))dA 

= ^ / HtdA+ [ 2fH2 + dHt{v)dA 

and eq. ( |l4[) follows, because SEt = —dHt (see (||)). Eq. (15) is a direct consequence because 
H^ = tr(iff) +2iJ2- 
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Taking twice the trace of the Gauss equation for Et shows that: 

S^=m{m-l)K + 2H2 , 

so that eq. ( [T^ ) becomes: 

mKV{Vi) = / S^dV ~ m{m - l)KV{n) + / HdA 
JQ Jan 

which proves (16). □ 

This leads for instance to the following simple consequence: 

Corollary 5 If K > 0, then no open domain of M has a foliation by closed, minimal hyper surf aces. 
If K = 0, any such foliation is by totally geodesic hyper surf aces. 

Proof: Suppose first that K > Q. Apply cq. (15) to such a foliation. The boundary term 
vanishes, and the right-hand side is therefore non-positive, while the left-hand side is positive, a 
contradiction. 

UK — 0, the same argument shows that II2 = 0, therefore iff = on each hypersurface by 
eq. (15), and each hypersurface is totally geodesic. □ 

This strongly contrasts with the negatively curved case; for instance, it is conjectured that a 
hyperbolic 3-manifold which fibers over the circle admits a foliation by compact minimal surfaces. 
Equation (15) indicates that such a minimal foliation should have a remarkable property: the 
Gauss curvature of each leave, integrated against a weight corresponding to the amplitude of the 
normal deformation, should be constant. 

Corollary 1^ is not too difficult to obtain by other methods; it is interesting to remark, however, 
that equations (|lj) and (15) can also be used to obtain more general results, for instance to give an 
integral lower bound on the mean curvature of a foliation by minimal hypersurfaces in a positively 
curved Einstein manifold. 



5 A quick tour of integral geometry 

In this section, we give a summary of some concepts of integral geometry which permit us to 
interpret some of our results more geometrically. Of course, it cannot be hoped that we can give 
anything resembling a comprehensive survey. A reader more interested in this fascinating subject 



is referred to the treatises of Santalo | 3an76 | and of Burago and Zalgaller [ BuZ |. 

First, we recall some formulas of Crofton type. Consider n-dimensional Euclidean space i?" and 
consider the Grassmanian of all the afhne m-planes in i?" - G"^ . This has a measure invariant with 
respect to the isometry group of i?". Any two such measures differ by a constant factor. There is a 
standard way to normalize, which will be implicit in the identities we shall state. Anyway, call the 
"canonical" invariant volume form dv^. There is a natural functional defined on (not necessarily) 
convex sets K in i?", to wit. 



Another natural functional is the following: Consider the space of m-dimensional linear subspaces 
of i?", and consider the average m-dimensional area of the projections of K onto such subspaces. 
This is the so-called Quermassintegral Wm{K). It is fairly clear that Pm and Wm are related, and 
indeed, one of the fundamental formulas of integral geometry ([San76, eq. (14.1)]) is 

(n - m)Om-i ■ • ■ OiOo 
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where Oi is just the surface area of the i-dimensional unit sphere S*' (it should be noted that the 
fraction in eq. (]l7| ) is just the volume of the Grassmanian of the m-dimensional subspaces in E"^). 

Equation (|17D allows us to relate the quantity Pm{K) to the integral of the m-th symmetric 
function of curvature, as follows: Consider the volume of the e neighborhood of K. A simple 
computation with radii of curvature shows that if k = ki,...,k„_i is the vector of principal 
curvatures of dK (we assume that dK is at least smooth), and <t„i{k) is the m-th symmetric 
function of curvature, then 

V,{K) = V{K) + J2 —-7 / '^^n{K). (18) 
On the other hand, there is another expression for Ve{K), due to Steiner (see [San76, III. 13. 3]): 



V.iK)^±ff)W. 

1— n V / 



iK)e\ (19) 



Comparing the coefficients of the powers of e in formulas (18) and (p^), we see that: 



m a™_i(i^)ds= ]Wm{K) 



dK 



m 



In other words, the integral mean curvatures of K are directly expressible in terms of the 
measures of the set of planes intersecting K, and the average projection measures. In particular, 
since (Tq is equal to 1, we see that the area of dK is equal to a constant factor times the measure 
of the set of lines intersecting K , while the total (first) mean curvature is a constant times the 
measure of the set of 2-planes intersecting K . Since we are especially interested in n = 3, we will 
write down the constants explicitly in that case: 

AidK) = 31^1 (i^), while Pi{K) = ^Wi{K), so 

A{dK) = -Pi{K). (20) 

TT 

On the other hand, Jg^iki + fc2) = SW2{K), while P2iK) = f M^2(^), so 

/ ih + k2) = 2P2iK). (21) 

JdK 

So far, we have talked about convex bodies in a Euclidean setting, but the theory can be 
extended to other symmetric spaces, in particular, to i?" and S"". The expressions become a 
bit more complicated in general, but in three dimensions, they are simple enough; the following 
formulas are in [San76, eq. (17.62)]: 



P,iK) = ^A{dK), P2{K) ^ \ I (fci + k2) + kV{K), (22) 

IdK 



where k is the sectional curvature of the ambient space (so the formula reduces to eq. (21) when 
fc = 0). It should be noted that there is yet another interpretation of the quantity P2{K), in terms 
of the polar map of a spherical or hyperbolic convex body (the spherical version is classical, the 
hyperbolic has been studied in the first author's thesis | RivH93| ): this map associates to K the 



set K* of hyperplanes intersecting K. For the sphere 5'", the set K* can be naturally viewed as a 
convex body in S*", for i7" the set of hyperplanes can be naturally viewed as the de Sitter space 



Alexandrov's inequality ([ BuZ , p. 145]) is the following: for a convex K in i?", the cross- 
sectional measures satisfy: 

V^{K)>vl'V^{K), j>i, (23) 



19 



with equality if and only if iiT is a ball in i?". Here Vi can be defined by the following formula: 



( PuZ| , p. 140]): 

where w„ is the volume of the n-dimensional unit ball ; k-i {i = 1, ... ,n — 1) are the principal 
curvatures of dK at the point x of dK , dF is the area element of dK, the sum is taken over all 
possible finite sequence of indices ji, . . . , j„-m-i- In particular, Vn-2{K) is related to Q{S) — H 
by Q{S) — n{n — \)Vn-2{K). Aleksandrov's inequality in can thus be restated as follows : 
Among all convex bodies with a fixed Pi, the ball has the biggest P2. Our Theorem 12 is exactly 
the extension of this result to other three-dimensional space forms. 



6 Extending the Aleksandrov inequality 

This section contains applications of the previous results in the simple setting of three-dimensional 
space- forms, esp. . Thus we consider a smooth, strictly convex surface E in a constant curvature 
space M which might be , R'^, or (in this case we suppose that S is space- like). 
To keep notations close to that of the previous section, we define a functional P2 as: 

2P2 := I Hda ~ 2€KqV , 

where Kq is the sectional curvature of M, and e = 1 if M is Riemannian, e = —1 if AI = Sf. 
Note that, for any deformation of E: 



J Hda^ = J H'da + ^J H{I',I)da. 



Therefore, as a consequence of equations and (|5|), we have for any deformation of E: 

P^ = -^JjI',II-HI)da (24) 

Consider a normal deformation of E, i.e. an infinitesimal deformation by a vector field /n, 
where n is the (exterior) unit normal to E. Then /' = 2fE, so that: 

2P^= I {-flI,F - HI)da= I 2fKeda (25) 

where := det{ff) is the extrinsic curvature of E. On the other hand, the area of E varies as: 

A' ^^Jj2fE,I)da^ J^fHda (26) 

As a consequence, we already find an extremely simple proof of a result with a fiavor of classical 
differential geometry. It can be seen as a consequence of some more general results (see [Ros88], 
and also [ EH89| ) but we include it here because of its extremely simple proof. 



Theorem 11 Suppose that E is a smooth, strictly convex surface in a 3-dimensional space-form, 
and that there exists a constant k G such that, on E, Kg = kH . Then E is totally umbilical. 

Proof: Suppose that Ke — kH. Then, by equations (26) and (25), E is a critical point of P2 
among surfaces with the same area. But it is well known (see [ Pog73| ) that all variations /' of / 



are induced by deformations of E. Therefore, eq. ( p,4\ ) shows that there exists a constant k' such 
that E — HI = k'l, so that E is totally umbilical. □ 

We now turn to the extension of the classical Alexandrov inequality (see Section ^) for convex 
surfaces in Euclidean space to three-dimensional space-forms. 
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Theorem 12 Let S be a compact convex surface in (resp. , S^). Let V{S) be the volume 
of the interior of S, and call 2P2{S) := ^gHda + 2V{S) (resp. 2P2{S) = JgHda, 2P2{S) = 
JgHda — 2V{S)). There exists a (unique modulo global isometrics) umbilical surface Sq with 
Area{So) = Area{S), and PaC^o) > P2iS). 

Remark 4 An elementary consequence is the well-known fact that a convex surface in has area 
at most 2tt. 

Proof. We will give the proof of Theorem |l^ in the hyperbolic case, the other two situations 
are very similar. For fcg > 0, let CA,ka be the space of smooth, convex surfaces in with area 
A and principal curvatures at most fco: and containing a given fixed point xq- It is again an 



elementary consequence of eq. (E4[) and of |Pog73 that the only critical points of P2 in CA.ko a-i'e 



the umbilical hypersurfaces (there is a unique such surface in Ca ka modulo the global isometries 
oiH^). 

It is therefore sufficient to prove that P2 has a maximum in the interior of CA,ko- This will be 
a consequence of the following points: 

1. if S* e CA.ko contains a point s where a principal curvature vanishes, then there exists an 
infinitesimal deformation of S increasing the minimum of the principal curvatures and in- 
creasing P2, while leaving the area constant; 

2. if fco > 1 and if 5 G CA.ko contains a point s where a principal curvature is equal to fco, 
then there exists a deformation of S decreasing the maximum of the principal curvatures and 
increasing P2; 

3. for each M > 0, there exists L > such that if S* G CA,ko ha-s (extrinsic) diameter above L, 
then P2{S)<-M. 

Theorem follows, because a maximizing sequence for P2 can neither "degenerate" (because of 
point (3)), nor converge to a surface with a vanishing principal curvature (point (1)) or a principal 
curvature equal to fco (point (2)). 

To prove point (1), note that the equation (^) simplifies here to: 

E' - Hf + fL-fM (27) 

Therefore, to insure that a deformation fn increases the minimum of the principal curvatures at 
a point where this minimum vanishes, it is enough to have: / > \\Hf\\ over S. But, if a principal 
curvature of S vanishes, then S is not umbilical, so that H and K^, are not proportional on S. It 
is then easy to check using equations (25) and (26) that there exists a normal deformation of S 
with the right properties. 

Point (2) can be proved in the same way, the condition on / is now that (fco ~ 1)/ > 11^/11- 
Finally, for point (3), let S* be a convex surface in , and, for e > 0, call £'e the set of points 
at distance at most e from the interior of S, and := dE^. Let Ti^ be the integral mean curvature 
of Se, and let Ae be its area and Ve the volume of its interior E^. Equation (0) shows that: 



.2^ = ^+1 {n,F-HI)da 
de de Jg^ 



so that: 

2^ + ^ = / Keda = f {K + l)da = 4^ + A , 
where we have used the Gauss-Bonnet theorem. Since dV^/de — Ae, we have: 

dVe ^ dHe _ 

de dt ' 
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and therefore: 

d^Ve dV, , 

Integrating this EDO leads to a classical formula for V^: 

V, = A sinh(e) + 47r(e - sinh(e)) + Ho(cosh(e) - 1) + Vq (28) 

Now suppose that S has extrinsic diameter at least L, then Eq contains a segment 7 of length at 
least L. Applying equation (28) to a sequence of convex surfaces in Eq which converges to 7, we 
find a lower bound for V^: 

V, > 47r(e - sinh(e)) + 27rL(cosh(e) - 1) 

so that, for any e > 0: 

Asinh(e) + 7i;o(cosh(e) - 1) + Vo > 27rL(cosh(e) - 1) 
Taking the limit as e — > 00 shows that: 

Ao+Hq> 2-kL 

and point (3) follows. 

This finishes the proof of Theorem □ 

Note that the "local" part of this argument extends partly to higher dimensions, again for 
Einstein manifolds with convex boundaries. Namely, if (Af, dM) is such a manifold, we say that 
it is "rigid" if it admits no infinitesimal Einstein deformation which does not change the induced 
metric on the boundary. It is proved in [ ^ch98 | that, in that case, all infinitesimal deformations 



of the metric on dM are induced (uniquely) by Einstein deformations of M. Therefore, it is still 
true in that case that, if (M, dM) is a critical point of P2 among Einstein metrics with the same 
"area" , then it is umbilical. We do not know whether there exists any non-rigid Einstein metric 
with negative curvature and strictly convex boundary. 
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